Move all job PROPERTIES to Jenkins Files for Build and Pipeline jobs #2138

AdamBrousseau · 2018-06-08T20:28:04Z

Add Build Discarder
- Retrieve values from Variable file
Add job Parameters
- Default values for Vendor Params must be set globally
  even if they are blank
No Trigger for Build and Pipeline jobs
TODO add trigger for PR builds

[skip ci]
Issue #1017
Signed-off-by: Adam Brousseau adam.brousseau88@gmail.com

Depends ~~#2093~~

AdamBrousseau · 2018-06-16T19:14:03Z

vsebe · 2018-06-19T13:37:58Z

buildenv/jenkins/jobs/builds/Build-Test-Any-Platform

+    // Keep last 50 of each PullRequest build
+    NUM_BUILDS = 50
+}
+properties([buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: NUM_ARTIFACTS, daysToKeepStr: DAYS_BUILDS, numToKeepStr: NUM_BUILDS))])


Missing strategy?

looks not to be mandatory

vsebe · 2018-06-19T14:13:24Z

buildenv/jenkins/jobs/builds/Build-Test-Any-Platform

+    // Keep last 50 of each PullRequest build
+    NUM_BUILDS = 50
+}
+properties([buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: NUM_ARTIFACTS, daysToKeepStr: DAYS_BUILDS, numToKeepStr: NUM_BUILDS))])


If you move this code into a fuction in variables-functions then you avoid duplicating the code.

AdamBrousseau · 2018-06-19T14:20:46Z

jenkins compile win jdk10

vsebe · 2018-06-19T14:19:42Z

buildenv/jenkins/jobs/builds/Build-Test-Any-Platform

@@ -20,6 +20,16 @@
 * SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0 WITH Classpath-exception-2.0 OR LicenseRef-GPL-2.0 WITH Assembly-exception
 *******************************************************************************/

+// Discard old builds
+NUM_ARTIFACTS = 5


move NUM_ARTIFACTS, NUM_BUILDS , DAYS_BUILDS to variables file

vsebe

reformat to use variable files

AdamBrousseau · 2018-06-19T14:23:48Z

jenkins compile win jdk10

AdamBrousseau · 2018-06-19T14:27:24Z

jenkins compile win jdk10

AdamBrousseau · 2018-06-19T14:31:25Z

jenkins compile win jdk10

AdamBrousseau · 2018-06-19T14:34:42Z

jenkins compile win jdk10

AdamBrousseau · 2018-06-19T14:37:31Z

jenkins compile win jdk10

AdamBrousseau · 2018-06-19T14:38:50Z

jenkins compile win jdk10

AdamBrousseau · 2018-06-19T15:20:41Z

Unfortunately having the properties step will overwrite the manually configured trigger property in the job even if that section isn't defined in the pipeline. I.e. We are trying to add build discarder but since trigger is also a property and its not defined in the code, the PRs trigger section will get obliterated upon the first run of the build.

This issue comment and the next 3 comments sum it up nicely.

See pipeline-syntax Sample Step: properties to see all the properties you can define. We currently use parameters, triggers, and build discarder. Since we set a different SPEC & PLATFORM parameter on every job, I don't this this will work as is.

We might be able to define everything in the jenkinsfiles. We would have to have all the parameters and all the trigger regexes and the user whitelist in the variable file, which is a good thing. Upon initial setup you could leave all the config blank, and we could have an if in the pipeline that first checked for PLATFORM and SPEC, if these were undefined we could parse the job name(#1396) to set them up. Then all the rest of the properties could be determined afterwards.

I will change this PR to WIP and update the title to reflect the larger task.

AdamBrousseau · 2018-08-29T16:02:15Z

Jenkins copyright check

vsebe · 2018-08-30T14:02:03Z

buildenv/jenkins/common/variables-functions

+            /* OPENJDK_REPO, OPENJDK_BRANCH, OPENJDK_SHA,
+            *  OPENJ9_REPO, OPENJ9_BRANCH, OPENJ9_SHA,
+            *  OMR_REPO, OMR_BRANCH, OMR_SHA,
+            *  VARIABLE_FILE, VENDOR_REPO, VENDOR_BRANCH, VENDOR_CREDENTIALS_ID */


could use a for loop

vsebe · 2018-08-30T14:05:31Z

buildenv/jenkins/common/variables-functions

+
+        // If SDK_VERSION parameter is already set, assume it is correct (helps with Sandbox builds)
+        // Else parse the JOB_NAME to determine the verison
+        if (params.SDK_VERSION) {


replace if-else with if statement:

SDK_VERSION = params.SDK_VERSION if (!SDK_VERSION) { //infer from job name ... }

vsebe · 2018-08-30T14:07:13Z

buildenv/jenkins/common/variables-functions

+        }
+        echo "SDK_VERSION:'${SDK_VERSION}'"
+
+        if (params.PLATFORM) {


replace if-else with if statement only

PLATFORM = params.PLATFORM if (!PLATFORM) { ... }

vsebe · 2018-08-30T14:10:44Z

buildenv/jenkins/common/variables-functions

+    VENDOR_REPO = params.VENDOR_REPO
+    VENDOR_CREDENTIALS_ID = params.VENDOR_CREDENTIALS_ID
+    if (VARIABLE_FILE && VENDOR_REPO && VENDOR_BRANCH) {
+        echo "VARIABLE_FILE:'${VARIABLE_FILE}'"


single quotes not needed in echo statements

vsebe · 2018-09-06T20:11:27Z

buildenv/jenkins/common/variables-functions

+        echo "VENDOR_REPO:'${VENDOR_REPO}'"
+        echo "VENDOR_BRANCH:'${VENDOR_BRANCH}'"
+        echo "VENDOR_CREDENTIALS_ID:'${VENDOR_CREDENTIALS_ID}'"
+        if (VENDOR_CREDENTIALS_ID != '') {


if (VENDOR_CREDENTIALS_ID != '') replace with if (VENDOR_CREDENTIALS_ID)

vsebe · 2018-09-06T20:27:18Z

buildenv/jenkins/common/variables-functions

+            // PLEASE NOTE: The following 4 DEFAULT variables must be configured in the Jenkins Global Config (even if they are set to blank).
+            // This allows us to know what the default values are without being told explicitly.
+            // These DEFAULT values are typically constant per Jenkins Server.
+            PARAMETERS.add(string(name: 'VARIABLE_FILE', defaultValue: "${VARIABLE_FILE_DEFAULT}", description: '', trim: true))


add checks for the *_DEFAULT variables, e.g

if (VARIABLE_FILE_DEFAULT) { PARAMETERS.add(string(name: 'VARIABLE_FILE', defaultValue: "${VARIABLE_FILE_DEFAULT}", description: '', trim: true)) }

AdamBrousseau · 2018-09-07T18:14:53Z

@smlambert for final review

smlambert

Other than minor typo, LGTM

smlambert · 2018-09-07T18:32:10Z

buildenv/jenkins/common/variables-functions

+    if (JOB_TYPE == "build" || JOB_TYPE == "pipeline") {
+        // Start with just supporting Build and Pipeline jobs
+        // PR jobs will need some playing to get the trigger figured out
+        // Also regaring PR builds, see OpenJ9 Issue #2728


spelling regaring -> regarding

- Add Build Discarder - Retrieve values from Variable file - Add job Parameters - Default values for Vendor Params must be set globally even if they are blank - No Trigger for Build and Pipeline jobs - TODO add trigger for PR builds Issue eclipse-openj9#1017 [skip ci] Signed-off-by: Adam Brousseau <adam.brousseau88@gmail.com>

- Using params.PARAM will allow null to be passed to a downstream job. This causes a failure if the downstream job has the parameter configured. - Null value not allowed as an environment variable - Adding a check and setting the variable to blank should resolve this failure [skip ci] Issue eclipse-openj9#2138 Signed-off-by: Adam Brousseau <adam.brousseau88@gmail.com>

- Add BUILD_NODE, TEST_NODE, PERSONAL_BUILD and SLACK_CHANNEL - Missed in original PR eclipse-openj9#2138 [skip ci] Signed-off-by: Adam Brousseau <adam.brousseau88@gmail.com>

AdamBrousseau force-pushed the add_build_discarder branch from c2c5786 to c8dfa6d Compare June 8, 2018 20:47

pshipton added the comp:build label Jun 12, 2018

AdamBrousseau force-pushed the add_build_discarder branch from c8dfa6d to 238eb65 Compare June 16, 2018 19:12

AdamBrousseau changed the title ~~WIP: Add build discarder to generic pipelines~~ Add build discarder to pipelines Jun 16, 2018

vsebe approved these changes Jun 19, 2018

View reviewed changes

vsebe reviewed Jun 19, 2018

View reviewed changes

vsebe suggested changes Jun 19, 2018

View reviewed changes

AdamBrousseau force-pushed the add_build_discarder branch from 238eb65 to 5651298 Compare June 19, 2018 14:31

AdamBrousseau changed the title ~~Add build discarder to pipelines~~ WIP: Move all job PROPERTIES to Jenkins Files Jun 19, 2018

jdekonin mentioned this pull request Jul 25, 2018

Unable to launch jenkins build/sanity jobs from PR ibmruntimes/openj9-openjdk-jdk10#50

Closed

AdamBrousseau force-pushed the add_build_discarder branch 9 times, most recently from 7baa8d7 to ed3aa83 Compare July 27, 2018 19:21

AdamBrousseau mentioned this pull request Aug 29, 2018

WIP: Add properties to prs #2744

Closed

AdamBrousseau force-pushed the add_build_discarder branch 8 times, most recently from 051c4ab to a1511a0 Compare September 4, 2018 19:39

vsebe reviewed Sep 6, 2018

View reviewed changes

AdamBrousseau force-pushed the add_build_discarder branch 4 times, most recently from e0b99a5 to 8fafa7b Compare September 6, 2018 20:53

vsebe approved these changes Sep 7, 2018

View reviewed changes

smlambert approved these changes Sep 7, 2018

View reviewed changes

AdamBrousseau force-pushed the add_build_discarder branch from 8fafa7b to 2d3379d Compare September 7, 2018 18:39

smlambert merged commit b9aecfb into eclipse-openj9:master Sep 7, 2018

AdamBrousseau mentioned this pull request Sep 10, 2018

Don't pass params directly to downstream jobs #2820

Merged

AdamBrousseau mentioned this pull request Sep 26, 2018

Add missing params to build and pipeline jobs #3042

Merged

AdamBrousseau mentioned this pull request May 6, 2019

Convert PullRequests to pipeline of jobs #2836

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move all job PROPERTIES to Jenkins Files for Build and Pipeline jobs #2138

Move all job PROPERTIES to Jenkins Files for Build and Pipeline jobs #2138

AdamBrousseau commented Jun 8, 2018 •

edited

Loading

AdamBrousseau commented Jun 16, 2018

vsebe Jun 19, 2018

vsebe Jun 19, 2018

vsebe Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

vsebe Jun 19, 2018

vsebe left a comment

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018 •

edited

Loading

AdamBrousseau commented Aug 29, 2018

vsebe Aug 30, 2018

vsebe Aug 30, 2018

vsebe Aug 30, 2018

vsebe Aug 30, 2018

vsebe Sep 6, 2018

vsebe Sep 6, 2018

AdamBrousseau commented Sep 7, 2018

smlambert left a comment

smlambert Sep 7, 2018

AdamBrousseau Sep 7, 2018

Move all job PROPERTIES to Jenkins Files for Build and Pipeline jobs #2138

Move all job PROPERTIES to Jenkins Files for Build and Pipeline jobs #2138

Conversation

AdamBrousseau commented Jun 8, 2018 • edited Loading

AdamBrousseau commented Jun 16, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AdamBrousseau commented Jun 19, 2018

Choose a reason for hiding this comment

vsebe left a comment

Choose a reason for hiding this comment

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018

AdamBrousseau commented Jun 19, 2018 • edited Loading

AdamBrousseau commented Aug 29, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AdamBrousseau commented Sep 7, 2018

smlambert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AdamBrousseau commented Jun 8, 2018 •

edited

Loading

AdamBrousseau commented Jun 19, 2018 •

edited

Loading